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ABSTRACT: 

PARTICIPATORY METHODS 

Participatory methods should be an integral part of any impact assessment for 
enterprise development. Their use is necessary to addressing the concerns of both 
the sustainable livelihoods approach and the human rights approach in DFID-funded 
enterprise interventions. Participatory methods are now well developed in relation to 
project-level impact assessment. 
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INTRODUCTION 

This paper gives an overview of issues and outlines the questions which need to be 
asked before deciding when to use participatory methods, who should participate 
and which particular techniques are most appropriate: 

Section 1: What are participatory methods? Principles and techniques gives 
an overview of different types of participatory methods and some recent 
innovations. 

Section 2: Using participatory methods: advantages, challenges and ways 
forward discusses ways in which the potential advantages of using 
participatory methods can best be realized. 

Section 3: Participatory methods and integrated impact assessment: 

guidelines for commissioning participatory assessments gives 
guidelines for commissioning participatory assessments in different 
types of enterprise intervention. 
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The paper provides guidelines for integrating participatory methods in impact 
assessment of different types of enterprise intervention. It does not give detailed 
step-by-step practical instructions on how to use specific participatory tools. For this 

the reader is referred to a number of manuals given in the additional resources at the 
end of the paper. 

Participatory methods should be an integral part of any impact assessment for 
enterprise development. Their use is necessary to address the concerns of both the 
sustainable livelihoods approach and the human rights approach in DFID-funded 
enterprise interventions. This is because: 

Complexities of livelihoods and poverty need to be understood in order to 
decide WHAT is to be assessed. Grassroots participation leads to more relevant 
identification of impact goals and measurable indicators. 

Different stakeholders are affected by enterprise development in different 
ways. Participatory methods enable better identification of WHO is affected in 
which ways. In particular they enable the voices of the very poor, women, 
children and vulnerable groups to be heard. 

Complexities of development processes need to be understood in order to 
analyse WHY particular impacts are occurring. Participatory methods enable 
complex interactions between contexts, grassroots aspirations and strategies, 
institutional structures and enterprise interventions to be better understood. 

Communication between donors, policymakers, development practitioners and 
those affected by interventions is needed to identify HOW POLICY CAN BE 
IMPROVED. Participatory methods facilitate realistic identification of the 
practical implications of the findings of impact assessment through negotiation 
between different stakeholders. 

Participatory methods are now well developed in relation to project-level impact 
assessment. In areas like micro-finance, enterprise training and fair trade impact 
assessments participatory methods have been used as part of: 

external donor impact assessment. A number of DFID-funded impact 
assessments have incorporated participatory tools as have the CGAP AIMS 
micro-finance assessments 1 . 

programme monitoring and evaluation. Micro-finance, fair trade and training 
programmes have been developing methods for participatory monitoring 
evaluation integrated into the ongoing activities of programme staff and existing 
Management Information Systems 2 . 



1 Examples funded by DFID include Kuapa Kokoo and FINCA Malawi. For examples in the AIMS 
studies see SEEP 2000 available on http://www.ids.ac.uk/cgap/ 

2 For micro-finance see Small Enterprise Foundation in South Africa http://www.sef.co.za CARE- 
PULSE in Zambia and MicroSave Africa. Participatory methods were extensively used in Oxfam's 
impact assessment of its Fair Trade partners. For a general overview of issues in participatory 
monitoring evaluation see Estrella et al eds 1998. 



There is also increasing experience of using participatory methods in: 



. macro level policy assessment. Examples of macro level assessments co- 
funded by DFID in which participatory methods have been used include 
participatory poverty assessments 3 , HIV/AIDS policy assessments 4 and health 
and education policy research. 5 Other examples include the ILO and UNIFEM- 
funded WIEGO 6 research on women and policy for the informal sector. 

However despite the important potential contribution of participatory methods, their 
use needs to be well thought through at all stages of the impact assessment. They 
cannot be seen as an easy or cheap option. Nor can they be seen simply as a set of 
tools which can easily be tacked onto conventional quantitative impact assessment 
in order to give the appearance of grassroots participation. Importantly, although 
participatory methods can be used on their own for rapid exploratory assessment, in 
order to increase their reliability and credibility they need to be triangulated with 
other quantitative and qualitative methods as part of an integrated impact 
assessment. 



3 See Brocklesby and Holland 1998; Booth et al 1998; Holland and Blackburn 1998. 

4 See UK NGO AIDS Consortium 1996. 

5 For health see papers in Cornwall et al eds. 2000. For education see McKay and Treffgarne 2000. 

6 Women in Informal Employment Globalizing and Organizing WIEGO . 



1 : WHAT ARE PARTICIPATORY METHODS? PRINCIPLES AND TECHNIQUES 



The participatory methods discussed here are now generically referred to as 
Participatory Learning and Action (PLA). Participatory Learning and Action 
combines: 

a set of diagramming and visual techniques originally developed for 
livelihoods analysis and now widely used in Natural Resources departments in 
development agencies. They have since been adapted for use in other sectors 
including enterprise development. 

. underlying principles of grassroots participation from human rights activism 
which involve rethinking power relations and partnerships between development 
agencies, experts and poor people. These are now being developed further to 
facilitate negotiations between different stakeholders in projects and policy 
dialogue. 

The underlying principles and most common techniques are summarised in Box 1. 

Participatory methods for impact assessment are most commonly associated with 
the spread of diagramming and visual techniques which began to be developed in 
the 1970s. These originated in a number of scientific disciplines interested in 
analysis of complex systems: biological science, ecology, agricultural economics and 
geography. It became increasingly important to work with farmers to develop more 
sophisticated models to explain their responses to development programmes. The 
increasing influence of applied anthropology in development agencies from the 
1980s also led to greater awareness of the need for a more sophisticated 
understanding of poverty, social processes and grassroots perspectives on 
development. By the end of the 1980s diagramming techniques bringing together the 
insights from these different disciplines had been combined into a flexible 
methodology commonly referred to as Rapid Rural Appraisal (RRA). Parallel to these 
developments in the South, methodologies like Soft Systems Analysis and Cognitive 
Mapping also became increasingly common in areas like management consultancy, 
organizational research and planning. Here diagrams were used for institutional 
analysis, highlighting problem areas and brainstorming possible solutions. 
Workshops for organizations and enterprises included senior executives and 
managers. 

By the mid 1990s it was becoming increasingly evident that the mechanical 
application of these techniques was often failing to really reach and capture the 
views of poor people, particularly women, children and socially excluded. There was 
renewed interest in methodologies for participation drawing on earlier traditions 
of participatory action research which had been long established as an integral part 
of many grassroots organisations in the South. In India for example SEWA and 
other women's organisations based their programmes on the findings of focus group 
discussions in the 1970s and 1980s. MYRADA and AKRSP developed 



BOX 1: PLA : UNDERLYING PRINCIPLES AND TECHNIQUES 



UNDERLYING PRINCIPLES 

. embracing complexity 

recognition of multiple realities 
. prioritising the realities of the poor and disadvantaged 

grassroots empowerment 

from assessment to sustainable learning 

relating learning to action 

TECHNIQUES 
Visual Techniques 

Diagrams: Flow/causal diagram; Venn/Institutional diagram; Systems diagrams; 
Pie charts; Histograms 

Ranking Techniques: Preference ranking and scoring; Pairwise ranking; Direct 
matrix ranking; Ranking by voting; Wealth ranking 

Time Trends Analysis: Historical and future (visioning) mapping; Time trends 
charts; Oral Histories 

Mapping Techniques: Mobility mapping; Social mapping; Transect (walks) 
Calendars: Seasonal calendar; Historical seasonal calendar 
Ethno-Classifications: Proverbs, Stories, Indigenous Categories and Terms, 
Taxonomies 

Group and team dynamics methods 
. Focus Group discussions 
. Role-play 

. Participatory workshops 
Recent Innovations 

. Photo and Video production with grassroots groups 

Grassroots diaries using diagrams and simple numerical and written 
information 

OTHER COMPLEMENTARY QUANTITATIVE AND QUALITATIVE METHODS OF 
INVESTIGATION 

• Secondary Data Review 

• Structured questionnaires 

. Semi-structured interviewing 
. Case studies 

• Participant observation 
. Direct observation, 

• Qualitative anthropological fieldwork 

participatory methodologies for grassroots mobilization'. In Latin America Paolo 
Freire developed methodologies for action research which were later adopted in 
other countries. These methodologies were based on underlying principles of 
human rights and aimed to use the research process itself as a means of 



7 See for example PALM manuals on MYRADA website Myrada. .. 



empowerment through use of diagrams as a focus for discussion and other 
methodologies like drawing and role-play. More recently these methodologies have 
been extended to include photography and video by grassroots groups and looking 
at ways in which grassroots groups can record their own information in diaries. 

Initially the term PRA (Participatory Rural Appraisal) was used to describe the 
bringing together of RRA and activist research. It was emphasized that the most 
important aspect was not the diagramming tools but their flexible application based 
on a number of underlying principles: 8 

embracing complexity and seeking to understand it rather than oversimplifying 
reality in accordance with predetermined categories and theories 

recognition of multiple realities to be taken into account in analysis or action. 

prioritising the realities of the poor and most disadvantaged as equal 
partners in knowledge creation and problem analysis. 

grassroots empowerment, aiming not only to gather information about impact, 
but to make the assessment process itself a contribution to empowerment 
through linking grassroots learning and networking into policy-making. 

More recently the term Participatory Learning and Action (PLA) has become 
preferred because it more effectively incorporates the underlying human rights 
tradition through emphasising the importance of: 

changing from appraisal to learning and hence moving away from the use of 
participatory methods as an extractive process by outsiders to a sustainable 
learning process involving different stakeholders as equal partners. 

the importance of relating learning to action incorporating programme and 
policy improvement as an integral part of the learning process. 

Participatory methods are therefore a diverse and flexible set of techniques for visual 
representation and stakeholder involvement characterized by a set of underlying 
ethical principles. There is no one set of techniques to be mechanically applied in all 
contexts for all participants. There is on the one hand a set of visual tools to be 
flexibly applied to assist the synthesis and analysis of information which can be used 
in group settings and also as part of individual interviews. On the other hand are a 
set of guidelines for facilitating participation and negotiation in focus group 
discussions and workshops bringing together different stakeholders. These may or 
may not make substantial use of visual techniques. Generally both visual techniques 



For an excellent overview of the antecedents and development of PRA see the series of articles by 
Robert Chambers: 1994a,b,c. Here Chambers describes PRA as: "a growing family of approaches 
and methods to enable local "rural or urban' people to express, enhance, share and analyze 
their knowledge of life and conditions, to plan and to act' (Chambers 1994a). 



and participatory facilitation are combined in different ways. The emphasis is on 
innovation and creativity in adapting previous practice to new contexts and needs. 

2: USING PARTICIPATORY METHODS: ADVANTAGES, CHALLENGES AND 
WAYS FORWARD 

2.1 Participatory methods are now well developed in relation to project-level 
impact assessment. PLA methods, both diagramming techniques and group 
methodologies are increasingly used in both microfinance and fair trade impact 
assessments. They also have considerable potential for adaptation in other project- 
level assessments including BDS interventions such as training and marketing 
support. The participatory assessments which are now an established part of macro 
level policy assessment in other sectors like health and education could be adapted 
to macro level policy assessment of enterprise development. 

Participatory methods have the potential to bring together information from a 
diversity of sources more rapidly and cost effectively than quantitative or qualitative 
methods alone. As indicated in Box 2 they have a number of potential key 
contributions in increasing: 

Relevance of impact goals and indicators 

Stakeholder representation 
. Reliability of understanding of development processes 
. Credibility of practical inference 

A number of significant contributions to poverty assessment have been noted in 
reports on the recent World Bank-funded participatory assessments (seeee Box 3). 

However participatory methods also face a number of inherent challenges which 
need to be taken into account. Some of these are common to all impact assessment 
methodologies, some are due to the visual tools and some to the participatory 
process. The degree to which participatory methods realise their potential 
contribution depends critically on how carefully they are used and in what context. As 
noted above, participatory methods are not a fixed set of mechanistic tools but a 
diverse range of possible techniques which need to be flexibly adapted to particular 
situations and needs. In some cases problems can be resolved through innovation in 
the methods themselves. Sometimes limitations can only be , others can only be 
addressed through triangulation with other quantitative and qualitative methods. 



BOX 2: PARTICIPATORY METHODS: ADVANTAGES, CHALLENGES AND 


WAYS FORWARD 




Advantages 


Challenges 


Ways forward 


Relevance of 
impact goals 
and Indicators 


. rapidly 

identifying the range 
of potential impacts 

. participatory 
prioritisation of 
different impact 
goals 

. identification of 
locally relevant 
indicators 


. standardization 
of impact goals and 
indicators to allow 
comparative 
assessment 

. ensuring that 
sensitive issues are 
aired 


. using the same 
impact goals, 
weighting locally- 
specinc indicators 

. role play and/or 
triangulation with 
qualitative methods 


Representation 
of different 
stakeholders 


. identifying 
relevant stakeholder 
categories for 
assessment, control 
groups ana analysis 

. involving 
different 

stakeholders in a 
participatory 
process, including 
the most vulnerable 


. the focus on 
consensus may 
mask differences 

. pnci irinn th^it thf^ 
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most vulnerable are 
present and their 
voices are heard 

. resolving 
differences between 
stakeholders 


. paying attention 
to participatory 
process: location, 
timing, composition 
OT aiscussion groups 
and discussion 
agenda 

. triangulation with 
quantitative survey 
or informal 
qualitative targeted 
interviews 


Reliability of 
findings 


. rapidly obtaining 
impact and other 
information for 
whole communities 
or groups 

. exploring 
processes and 
hypotheses 

. rapidly 
investigating 
underlying or side 
issues 


. scale and 
representation 
beyond physically 
identifiable 
communities 

. focus on 
diagram outputs 
may detract from 
analysis of 
processes 


. using 
quantitative PLA 
methods eg 
mapping and voting 

• careiui 
documentation of 
context and the 
assessment process 

. triangulation with 
other methods 


Credibility of 


. increases 


. may raise 


. attention to 



practical 


communication 


unrealistic 


identifying and 


inference 


between donors, 


expectations 


clarifying the 




policymakers, 




limitations of the 




development 


. may create 


programmes and 




nrartitinnprs and 


tensions which 


nnlirips 




those affected by 


cannot be resolved 
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careful attention 








to the participatory 




. makes 




process 




information 








immediately 








accessible to 








different 








stakeholders 







2.2 WHAT is to be assessed? Relevance of impact goals and indicators 



Enterprise interventions funded by DFID have the primary stated goal of poverty 
reduction. This involves a range of different complementary types of intervention 
from support for large and medium-scale private sector development to direct 
support for small and micro-enterprises and trade unions. Some impact assessments 
may aim to assess the poverty impact of one intervention in one context as part of 
project management. Others aim to compare the relative effectiveness of different 
types of intervention and/or across different types of context and/or for different 
target groups. This latter is becoming increasingly important as donors attempt to 
develop coherent sectoral policies to increase the impact of their support. 

In either case the identification and prioritsation of impact goals to be assessed is 
contentious. Impact assessment based on market incomes of direct project 
participants or those targeted by macro-level policies is likely to miss a range of 
other positive and negative impacts. Examples of key issues which are likely to be 
missed are: 

. qualitative aspects of working conditions and ways in which incomes are earned 
which have implications for health, nutrition, skills acquisition for future increases in 
income and vulnerability to crises 

. intra-household relations, particularly gender and age inequalities, which have 
implications for the relative well-being of different household members 

. broader impacts on inequality, environment, civil society and democratization 
which affect access to services and resources, security for investment and so on. 



9 In this paper the term 'impact goal' is used to refer to what is to be assessed eg increased income, 
the term 'indicator' is used to refer to how it is to be assessed or measured eg increase in enterprise 
profits, increase in wages, increase in consumption of particular goods 



BOX 3: PARTICIPATORY POVERTY ASSESSMENT: CONTRIBUTIONS OF 
PARTICIPATORY METHODS 

HIGHLIGHTING CONCEPTUALIZATIONS OF POVERTY 

Rural conceptualizations of poverty likely to be conceived of in terms of 

Community level: physical isolation, access to safe water, quality of land, social 

capital 

Individual level: ascribed attributes: gender (particularly widows, single mothers 
and to a lesser extent female heads of household often perceived as the 
poorest); age with the elderly being very poor; childlessness; health status and 
disability; hunger and nutrition; access to productive land; access to productive 
assets like livestock; access to health and education. 

Urban conceptualizations more likely to be individual in terms of individual 

ascribed status, particularly gender; income and employment; access to social 
and economic infrastructure 

Common issues: seasonality, corruption, political exclusion, discrimination in 
access to health and education services 

HIGHLIGHTING DIFFERENCE 

The PPAs highlighted a number of womens' concerns to be fed into policy: 

. relationship between polygamy and poverty 

. importance of women's tenancy rights 

. problems of increasing violence and conflict in the home 

IMPROVING UNDERSTANDING AND COMMUNICATION 

A key contribution to the subsequent policy dialogue was the awareness-raising 
function of the participatory techniques which in many cases was the first real direct 
exposure which some senior policy makers had had to the problems of poverty. 

POLICY CHANGES 

These varied depending on issues identified and the adequacy or otherwise of 
structures for feeding the findings into the policy-making process. The changes 
included: 

. greater emphasis on physical infrastructure like roads 

. greater emphasis on public services and attitudes of public service providers 

• greater attention to exposing and discussing corruption 

. greater emphasis on gender issues. 

Sources: Brocklesby and Holland 1998; Holland and Blackburn eds 1998 particularly 
papers by Robb and Norton. 



These are all important dimensions of poverty reduction as 
perceived by poor people (see Box 3) and are therefore also 
important to improving projects and/or gaining a balanced view of 
differences in impact between different types of intervention. 



At the same time most impact assessments cannot feasibly include all possible 
impact goals in a quantitative survey or qualitative investigation because of resource, 
time and/or skill constraints). Using participatory methods provides a useful way of: 

rapidly identifying the range of potential impacts through exploring the 
interlinkages between different dimensions of livelihoods and poverty 

prioritising the different possible impact goals for assessment by identifying 
which of the range of possible impacts are most important for the primary 
stakeholders eg increased income versus decreased leisure, levels of income 
and/or working conditions expected 

identifying reliable measurable or qualitative indicators which can be used 
in different contexts or for different target groups. 

Commonly used techniques include: 

. flow diagrams and Venn diagrams to identify the range of possible 
interlinkages between the different dimensions 

seasonal calendars to identify particular periods of high work activity and/or 
income vulnerability. 

. wealth ranking and other ranking techniques to identify the ways in which 
people define poverty and prioritise different impact goals and identify measurable 
indicators 

These potential contributions of participatory methods also have their own 
challenges: 

challenge of standardisation for comparison because of the emphasis on 
locally-identified impact goals and indicators there are added challenges for 
comparative assessment. It is however possible to agree on common impact 
goals to be applied across contexts and then weight locally-specific indicators by 
which they are to be measured. This is merely an extension of the weighting 
procedure itself which inevitably involves a certain amount of rather imprecise 
measurement and assumptions. The participatory process enables the rather 
arbitrary nature of a priori weighting to be based on local rather than external 
priorities. 

challenge of sensitive information: it cannot be assumed that the participatory 
process will necessarily yield in-depth information on sensitive issues. In some 
cases the inevitably unpredictable nature of participation has enabled generally 
private issues like domestic violence or controversial issues like caste 
discrimination to be raised. Including techniques like role play has been found 
useful in highlighting dimensions of power relations. In other contexts this sort of 
sensitive information may require in-depth investigation by qualitative methods. 



2.3 WHO is to be included? Questions of representation 

Enterprise development interventions affect different people in different ways. Those 
affected by interventions include not only primary stakeholders (e.g. entrepreneurs 
involved in training or credit programmes or employees in assisted enterprises) but 
also secondary stakeholders including: 

. other family members 

. other entrepreneurs in the same markets 

. consumers 

. staff in development agencies 

. others affected by macro-level change. 

The list of potential stakeholders is often very long. Moreover each stakeholder 
category is often an umbrella for different sub-categories e.g. by gender, age, 
income level, ethnicity and so on. Assuming homogeneity of 'communities' or 
'beneficiaries' is particularly problematic. As indicated in Box 3 the participatory 
poverty assessments have highlighted differences between women and men, and 
also between different age groups and between different ethnic groups and 
economic classes. At the beginning of an assessment it is often unclear which 
stakeholder categories are most relevant for the issues concerned. A controversial 
example is gender - for example, should women be treated as a separate category? 
should all women be placed in the same category? Identification of stakeholder 
categories is crucial, particularly for non-random sampling, choice of control groups 
and statistical analysis. At the same time, including a very broad range of 
stakeholder categories in the sample for impact assessment as well as control 
groups is often extremely costly and time-consuming. For this reason categories are 
generally simplified based on (often rather arbitrary) a priori assumptions, frequently 
leading to oversimplified and/or biased conclusions. 

The participatory techniques outlined above can again be used to make the process 
of simplification less arbitrary. The diagramming and ranking techniques can be used 
to: 

. refine stakeholder categories. 

. give visibility to the voices of the most disadvantaged as a distinct set of 
interests 

However again the participatory process has a number of inherent challenges: 

• emphasis on consensus may serve to privilege dominant views and further 
marginalise the most disadvantaged through giving the appearance of participation. 
Explicit attempts are likely to be needed to include the very poor, women and socially 
excluded and also to ensure that they are not only present, but their voices are 
heard. This requires careful attention to the participatory process: who participates 
which may require particular attention to location and timing, how they participate 
which may require separate meetings or special allocation of time to particular 
participants. 



• the differences and also potential conflicts of interest may be difficult to 
resolve and may require careful skills of negotiation. Recognising and 
addressing these differences and potential conflicts is nevertheless still essential 
to ensuring that the impact assessment is a reliable representation of contribution 
to poverty reduction. 



2.4 WHY are particular impacts are occurring? Reliability of findings 

Participatory methods are therefore useful at the initial stages of an impact 
assessment for identifying impact goals, indicators and stakeholder categories for 
sampling. Participatory methods can also be used for impact assessment itself to 
obtain a rapid visual overview for a broad range of impacts. Methods commonly 
used include: 

• mapping techniques including for example wealth and/or social mapping of 
specific communities or localities. Credit mapping can identify the extent of poverty 
targeting all social inclusion of credit programmes. Market mapping can identify 
types of enterprises, access to services and impacts. Controlled experiments 
comparing the use of these techniques with quantitative surveys have found them at 
least as accurate if not more so and much quicker and more cost-effective 
(Chambers 1997:141-146). Mapping techniques can include before and after data to 
get a rapid overview of the impact of a particular project or macro level policy. These 
can either compare a baseline and post-project mapping, or be based on recall. 
Participatory methods based on recall are likely to be more reliable than individual 
survey or qualitative information based on recall because it is possible to identify and 
investigate discrepancies in memory between people in groups. 

. voting techniques in group or community meetings or workshops. These can 
be open or secret ballot and can use visual as well as written questionnaires. Voting 
techniques can be used effectively in for example group based credit programmes, 
in end of training evaluations or community meetings to discuss the impact of macro 
level policy. They can be used to identify which people among the participants have 
been affected in which ways, or to reach a broader consensus on estimates of 
impact. 

• time trends analysis and seasonal calendars can be used to identify important 
dates for comparison and compare changes in seasonal dimensions of poverty and 
impact. 

drawing, role play, photography, video and grass-roots diaries also have a 
useful role to play here. 

Quantitative techniques are frequently inadequate to understand causal processes 
and many qualitative techniques conducted at the individual level are limited in their 
coverage. Participatory methods are very useful for investigating development 
processes and complex interactions between grass-roots perceptions and strategies, 
institutions and interventions. Commonly used techniques for analysing both context 
and interventions include: 



. diagram techniques like flow diagrams to explore causal processes, Venn 
diagrams to identify power relations and institutional analysis 

. ethno classifications and proverbs to explore attitudes and perceptions 
leading to particular outcomes 

Participatory methods have a particularly useful role in indicating issues to be further 
investigated through quantitative and qualitative techniques to ensure that these are 
cost-effectively targeted to specific hypotheses and/or the relevant persons. 
Participatory methods are also useful for investigating issues which emerge as 
relevant in the course of an assessment but which it is not possible to cover as a 
central point of investigation. 

There are nevertheless a number of limitations: 

. scale and representation beyond easily identifiable physical communities. 

The mapping and voting techniques can be applied when there are physical 
communities or groups which can be easily accessed or convened for the purpose. It 
is much more difficult to use these techniques where populations are scattered 
geographically or where user groups do not exist. It is possible sometimes to use 
other types of groups and networks e.g. neighbourhood associations which can act 
as a random sample with some users and some controls. Such situations may be 
better dealt with through other types of quantitative survey or informal qualitative 
targeted interviews. 

. skills, contextual knowledge and experience required to meaningfully 
record and analyse the visual outputs and participatory process. This is 
probably the greatest challenge of all for participatory methods. When badly 
conducted and recorded the visual outputs often appear arbitrary and meaningless, 
albeit rather quaint, to people who were not present during the meetings. This 
seriously jeopardises their reliability and credibility. These problems are not unique 
to participatory methods. Quantitative methods also oversimplify complex responses 
in order to fit them into predetermined boxes. Interviewer subjectivity inevitably 
influences the recording and analysis of both quantitative and qualitative information. 
In using participatory methods to some extent interviewer bias can be countered by 
the greater numerical strength of those interviewed and crossed checked by them. It 
is nevertheless crucial when using participatory methods to keep detailed notes on 
the process of investigation, how the findings compare with information from other 
sources, who participates and who does not and how particular diagrams are 
generated. Key questions to be documented are summarised in Box 4. 

Participatory methods have a useful contribution to make in analysis of the findings 
of impact assessment. They provide a useful means of investigating and 
crosschecking information gained by quantitative and qualitative methods at 
appropriate stages during the assessment. At the same time information obtained 
through participatory methods should also be cross-checked through triangulation 
with other methods to test their validity and increase their credibility. 



BOX 4: KEY ELEMENTS TO RECORD DURING PLA 

Objectives and organization of PLA exercises 
. Who initiated the PLA and why? 

What exercises were used? Why were these particular exercises selected in 

preference to others? For each what was their objective/s? 

Who conducted the exercsises? facilitators/researchers present; language 

issues (translation etc) 

Where were they conducted? Are any particular characteristics of the location 
significant? e.g. accessibility to different stakeholders, public space or private? 
At what time were they conducted? Date, time of day, duration of the exercise? 
Are any particular characteristics of the timing significant? e.g. accessibility to 
different stakeholders 

What materials were used? Are there any possible implications for their 
acceptability to different stakeholders? 

Recording the process 

. Who was present? (numbers, gender, poverty status, ethnicity, generation, 
names where appropriate, specific key individuals present) Who was not present? 
Do we know why? 

. For each exercise: who participated? How did they contribute to the discussion? 
Did this change during or between exercises? Were there any people who remained 
silent? Do we know why? 

. What exactly was the content of the discussion generated while the exercise was 

being carried out? What were the key quotations from participants? 

. Are there any points of interpretation essential to understanding the visual 

output? e.g. degrees of consensus, whether the diagram was jointly produced for the 

product of combining a number of separate diagrams, points of disagreement 

. Were any of the exercises changed to respond to circumstances? What and 

why? 

After the PLA exercise 

. Are there any relevant aspects of the context which are key to understanding the 
process of investigation? e.g. aspects of the social context known to be relevant, 
power relations. Do any of these require further investigation? 
. Which of the findings are considered reliable and credible in themselves? How do 
we know? 

. Which of the findings require further investigation by other methods? Which and 
how? 

. Which of the findings clearly indicate possibilities for improving the intervention/s 
under investigation? How can this be followed up? 



2.5 HOW can projects and policy be improved? Credibility of practical 
inference 

Quantitative and qualitative impact assessment frequently make inferences about 
the practical implications of their findings. However inferences cannot always be 
drawn from the information obtained, in which case recommendations are often 



based on the particular views of the researcher or in some cases of selected 
interviewees. Although these are often useful, they are rarely a systematic 
investigation of the feasibility of different options. 

Participatory methods have an important contribution in the systematic 
identification of realistic ways forward based on the information obtained from the 
impact assessment. As noted above, diagramming techniques are frequently used 
in management consultancy for this purpose. In addition the participatory process 
facilitates communication between donors, policymakers, development practitioners 
and those affected by interventions. The different stakeholders therefore both have 
their own opportunity to present their views and recommendations, and also to 
comment on those presented by other stakeholders. 

This participatory process needs to be carefully facilitated in order not to raise 
unrealistic expectations or create tensions which cannot be resolved. It is crucial 
that the potential limitations on change are clear to all those concerned, particularly 
in relation to resources available and the skills of development agencies. It is also 
important again to ensure fair representation of different stakeholders through the 
detail of focus group and workshop design. These issues are discussing detail in the 
manuals sited at the end of this paper. 

A particularly important contribution of participatory methods is their role in capacity 
building of the different stakeholders. Where sufficient attention is paid to this 
participatory methods have the potential to build up the necessary information 
resources and networks for a learning process which will be sustainable 
beyond the term of the one particular impact assessment. This can include. 

. Group-level learning 

. Ongoing particatory monitoring and evaluation 
. Multi-stakeholder networks for policy assessment 

Thus the costs of integrating participatory methods can be seen as a contribution to 
development in themselves, leading to much longer term benefits. These benefits 
have included reducing the costs of project administration, reducing default in micro- 
finance programmes, making training programmes more attractive to clients 
prepared to pay for services. This is therefore also a contribution to longer term 
financial sustainability of interventions. 



3: GUIDELINES FOR COMMISSIONING PARTICIPATORY ASSESSMENT 



Participatory methods have an important contribution to make at all stages of an 
impact assessment: 

. initial identification of impact goals, indicators, categories for sampling and 
analysis and initial exploration of hypotheses 

. during the impact assessment process for crosschecking and further 
exploration of issues raised by other methods 

. towards the end of the assessment to systematically identify and test the 
possible implications of any recommendations for improvement in projects or policy 
with different stakeholders 

Although there are inherent challenges faced by the use of participatory methods, 
these are no greater than for quantitative or qualitative methods. Much of the 
problem with participatory research in the past has been because of under 
resourcing, leading to mechanical use of tools by people with insufficient experience 
or understanding of context. Participatory methods cannot be seen as a cheap 
option. They must be treated as a serious and integral part of impact assessment 
requiring management by people with the skill and experience to flexibly adapt the 
different techniques to the particular issues, contexts and institutions being 
assessed. In many contexts a skilled facilitator would be able to train programme 
staff, local investigators and/or grassroots leaders to replicate selected simple 
exercises to extend the scope and coverage of the participatory investigation. This 
is particularly the case for longer term impact assessments and/or where at least 
some local people already have experience of participatory methods as co- 
facilitators or participants. Skilled participatory facilitators would also be able to 
identify any limitations in the information obtained and indicate ways in which the 
information obtained through participatory methods can best be triangulated with 
other quantitative and qualitative methods. 

The reliability and credibility in the use of participatory methods can be increased if 
those conducting the assessment are asked to carefully plan the assessment in 
advance, identifying which particular techniques are to be used, their aims, who is to 
be involved. There should also be an account of the potential risks and the ways in 
which the participatory process may need to be adapted to accommodate these. 
Although participatory methods will require to be flexible to circumstances and the 
contributions of participants, if there is a clear plan at the start it is easier to identify 
why changes were necessary and how these affect the assessment. It is also 
necessary to identify early on, and then at each successive stage, the limitations of 
the methodologies and how these are to be addressed through triangulation with 
other methods. A checklist of questions to be asked when commissioning a 
participatory assessment is given in Box 5. 



BOX 5: CHECKLIST OF QUESTIONS 
FACILITATORS 

What skills and experience do the facilitators have of using participatory 
methods? 

What experience do the facilitators have of the particular context and 
organisations involved? Do they have the required language skills? 

Is there a good representation of local facilitators? A good gender and ethnic 
balance? 

USE OF PARTICIPATORY METHODS 

Which participatory techniques are to be used at which stage? 
For each technique: 

. what are their objectives? How will they contribute to the relevance and reliability 
of the assessment? 

• who is to participate? How will participation contribute to representation, 
particularly of the disadvantaged? How will it contribute to or? 

How will the participatory process feed into project improvement and/or policy 
change? 

What are the potential risks? How will these be dealt with? 

What limitations of the participatory methods are foreseen? 

How are participatory methods to be integrated with quantitative and 
qualitative methods? 
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